Researchers examine if online physician reviews indicate clinical outcomes

In a new study, researchers from The University of Texas at Dallas investigated whether patient-generated online reviews of physicians accurately reflect the quality of care.

"Many patients use online reviews of physicians for deciding which physicians to see and which to avoid," said Dr. Atanu Lahiri, associate professor of information systems in the Naveen Jindal School of Management. "At the same time, though, some physicians have become quite wary of these websites, and they have even filed defamation lawsuits over negative patient reviews. In short, while patients seem to trust the reviews, the physicians don't."

In their study, published online Sept. 23 in Information Systems Research, Lahiri and his colleagues examined the relationship between online reviews of physicians and their patients' actual clinical outcomes. They wanted to know how much consumers can rely on the reviews, specifically in regard to chronic-disease care.

Prior research on online reviews of physicians is rather limited and does not cover chronic diseases in general, Lahiri said. According to the Centers for Disease Control and Prevention, 90% of the $3.5 trillion that the U.S. spends annually on health care goes toward patients with chronic conditions such as chronic obstructive pulmonary disease (COPD), diabetes and long-term mental-health issues.

"Any chronic disease, by definition, is treatable but not curable," Lahiri said. "Think of a diabetic patient who needs continuous care and whose disease will never fully subside. Contrast this with an acute-care service, such as surgery to fix a broken leg. If the patient can walk soon after the surgery, the patient would know the surgery went well and could write an online review of the surgeon. However, when there is no visible recovery cue, as with chronic diseases, how can patients possibly know how well doctors treated them?"

Add the fact that a person with a chronic disease may see several physicians over years, and that various social and economic factors oftentimes influence chronic-disease treatment outcomes, it becomes apparent that writing accurate reviews is even more challenging for a patient, Lahiri said.

"Naturally, one ought to ask, 'Are chronic-disease patients capable of writing reviews that can inform prospective patients about the true care quality provided by a physician?'" Lahiri said. "In other words, should we trust the reviews that we see on websites such as Vitals, Healthgrades and Yelp?"

For the study, the researchers examined 10 years of COPD patient admission-discharge data for hospitals in North Texas, tracking each patient's clinical journey spanning multiple physicians. They also studied online reviews of physicians on one of the ranking websites and scored the overall sentiments expressed in the text of each review in addition to considering the accompanying star ratings.

For chronic diseases, the study found that online reviews do not reliably indicate the quality of care provided by a physician, as measured in terms of readmission risk and other similar broadly accepted clinical outcomes. Both the star ratings and textual reviews were found to be equally uninformative of the actual quality of care, Lahiri said.

"The result was indeed a surprise," he said. "Since prior research on online reviews is mostly based on search goods and experience goods, it typically finds that online reviews are useful to prospective consumers. A key takeaway is that the efficacy of online reviews of search and experience goods does not extend to credence goods, such as chronic-disease care.

"Given that credence goods are inherently different from search or experience goods, it is important to study them separately, which is precisely what we set out to do in this research."

Lahiri recommends that consumers reduce their overreliance on physician-review sites. While reviews may provide some information, such as whether the staff is courteous, they are not necessarily reliable indicators of the quality of care.

The study also has implications for health care administrators. Some hospitals link physicians' compensation to the quality of patient care, Lahiri said.

"Our message to hospitals would be that online reviews of physicians do not necessarily provide the best proxy for quality, especially in the case of physicians treating chronic-disease patients," he said. "Also, patient surveys administered by hospitals may have similar issues, so hospitals should be very careful when evaluating physicians treating chronic-disease patients."

Dr. Zhiqiang Zheng, Ashbel Smith Professor of information systems and finance at UT Dallas; Danish H. Saifee PhD'18, assistant professor of management information systems at the University of Alabama; and Dr. Indranil R. Bardhan, the Foster Parker Centennial Professor in Information Technology at UT Austin, also contributed to the research.

During Saifee's doctoral studies at UT Dallas, Lahiri was his advisor, and Bardhan, who was then at UT Dallas, and Zheng served on his dissertation committee. An earlier version of the paper won a Best Student Paper award at the Conference on Health IT and Analytics in 2017.